GRADE guidance 37: rating imprecision in a body of evidence on test accuracy.

OBJECTIVES: To provide guidance on rating imprecision in a body of evidence assessing the accuracy of a single test. This guide will clarify when Grading of Recommendations Assessment, Development and Evaluation (GRADE) users should consider rating down the certainty of evidence by one or more levels for imprecision in test accuracy. STUDY DESIGN AND SETTING: A project group within the GRADE working group conducted iterative discussions and presentations at GRADE working group meetings to produce this guidance. RESULTS: Before rating the certainty of evidence, GRADE users should define the target of their certainty rating. GRADE recommends setting judgment thresholds defining what they consider a very accurate, accurate, inaccurate, and very inaccurate test. These thresholds should be set after considering consequences of testing and effects on people-important outcomes. GRADE's primary criterion for judging imprecision in test accuracy evidence is considering confidence intervals (i.e., CI approach) of absolute test accuracy results (true and false, positive, and negative results in a cohort of people). Based on the CI approach, when a CI appreciably crosses the predefined judgment threshold(s), one should consider rating down certainty of evidence by one or more levels, depending on the number of thresholds crossed. When the CI does not cross judgment threshold(s), GRADE suggests considering the sample size for an adequately powered test accuracy review (optimal or review information size [optimal information size (OIS)/review information size (RIS)]) in rating imprecision. If the combined sample size of the included studies in the review is smaller than the required OIS/RIS, one should consider rating down by one or more levels for imprecision. CONCLUSION: This paper extends previous GRADE guidance for rating imprecision in single test accuracy systematic reviews and guidelines, with a focus on the circumstances in which one should consider rating down one or more levels for imprecision.

Infectious Diseases Society of America Guidelines on the Diagnosis of COVID-19: Serologic Testing.

BACKGROUND: The role of serologic testing for SARS-CoV-2 has evolved during the pandemic as seroprevalence in global populations has increased. The Infectious Diseases Society of America (IDSA) convened an expert panel to perform a systematic review of the coronavirus disease 2019 (COVID-19) serology literature and construct updated best practice guidance related to SARS-CoV-2 serologic testing. This guideline is an update to the fourth in a series of rapid, frequently updated COVID-19 guidelines developed by IDSA. OBJECTIVE: To develop evidence-based recommendations and identify unmet research needs pertaining to the use of anti-SARS-CoV-2 antibody tests for diagnosis, decisions related to vaccination and administration of monoclonal antibodies or convalescent plasma in immunocompromised patients, and identification of a serologic correlate of immunity. METHODS: A multidisciplinary panel of infectious diseases clinicians, clinical microbiologists and experts in systematic literature reviewed, identified, and prioritized clinical questions related to the use of SARS-CoV-2 serologic tests. Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology was used to assess the certainty of evidence and make testing recommendations. RESULTS: The panel recommends against serologic testing to diagnose SARS-CoV-2 infection in the first two weeks after symptom onset (strong recommendations, low certainty of evidence). Serologic testing should not be used to provide evidence of COVID-19 in symptomatic patients with a high clinical suspicion and repeatedly negative nucleic acid amplification test results (strong recommendation, very low certainty of evidence). Serologic testing may assist with the diagnosis of multisystem inflammatory syndrome in children (strong recommendation, very low certainty of evidence). To seek evidence for prior SARS-CoV-2 infection, the panel suggests testing for IgG, IgG/IgM, or total antibodies to nucleocapsid protein three to five weeks after symptom onset (conditional recommendation, low certainty of evidence). In individuals with previous SARS-CoV-2 infection or vaccination, we suggest against routine serologic testing given no demonstrated benefit to improving patient outcomes (conditional recommendation, very low certainty of evidence.) The panel acknowledges further that a negative spike antibody test may be a useful metric to identify immunocompromised patients who are candidates for immune therapy. CONCLUSIONS: The high seroprevalence of antibodies against SARS-CoV-2 worldwide limits the utility of detecting anti-SARS CoV-2 antibody. The certainty of available evidence supporting the use of serology for diagnosis was graded as very low to low. Future studies should use serologic assays calibrated to a common reference standard.

The Infectious Diseases Society of America Guidelines on the Diagnosis of Coronavirus Disease 2019 (COVID-19): Molecular Diagnostic Testing.

Accurate molecular diagnostic tests are necessary for confirming a diagnosis of coronavirus disease 2019 (COVID-19) and for identifying asymptomatic carriage of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). The number of available SARS-CoV-2 nucleic acid detection tests continues to increase as does the COVID-19 diagnostic literature. Thus, the Infectious Diseases Society of America (IDSA) developed an evidence-based diagnostic guideline to assist clinicians, clinical laboratorians, patients, and policymakers in decisions related to the optimal use of SARS-CoV-2 nucleic acid amplification tests. In addition, we provide a conceptual framework for understanding molecular diagnostic test performance, discuss nuances of test result interpretation in a variety of practice settings, and highlight important unmet research needs related to COVID-19 diagnostic testing. IDSA convened a multidisciplinary panel of infectious diseases clinicians, clinical microbiologists, and experts in systematic literature review to identify and prioritize clinical questions and outcomes related to the use of SARS-CoV-2 molecular diagnostics. Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology was used to assess the certainty of evidence and make testing recommendations. The panel agreed on 12 diagnostic recommendations. Access to accurate SARS-CoV-2 nucleic acid testing is critical for patient care, hospital infection prevention, and the public health response to COVID-19 infection. Information on the clinical performance of available tests continues to grow, but the quality of evidence of the current literature to support this updated molecular diagnostic guideline remains moderate to very low. Recognizing these limitations, the IDSA panel weighed available diagnostic evidence and recommends nucleic acid testing for all symptomatic individuals suspected of having COVID-19. In addition, testing is suggested for asymptomatic individuals with known or suspected contact with a COVID-19 case when the results will impact isolation/quarantine/personal protective equipment (PPE) usage decisions. Evidence in support of rapid testing and testing of upper respiratory specimens other than nasopharyngeal swabs, which offer logistical advantages, is sufficient to warrant conditional recommendations in favor of these approaches.

Defining Misinformation and Related Terms in Health-Related Literature: Scoping Review.

BACKGROUND: Misinformation poses a serious challenge to clinical and policy decision-making in the health field. The COVID-19 pandemic amplified interest in misinformation and related terms and witnessed a proliferation of definitions. OBJECTIVE: We aim to assess the definitions of misinformation and related terms used in health-related literature. METHODS: We conducted a scoping review of systematic reviews by searching Ovid MEDLINE, Embase, Cochrane, and Epistemonikos databases for articles published within the last 5 years up till March 2023. Eligible studies were systematic reviews that stated misinformation or related terms as part of their objectives, conducted a systematic search of at least one database, and reported at least 1 definition for misinformation or related terms. We extracted definitions for the terms misinformation, disinformation, fake news, infodemic, and malinformation. Within each definition, we identified concepts and mapped them across misinformation-related terms. RESULTS: We included 41 eligible systematic reviews, out of which 32 (78%) reviews addressed the topic of public health emergencies (including the COVID-19 pandemic) and contained 75 definitions for misinformation and related terms. The definitions consisted of 20 for misinformation, 19 for disinformation, 10 for fake news, 24 for infodemic, and 2 for malinformation. "False/inaccurate/incorrect" was mentioned in 15 of 20 definitions of misinformation, 13 of 19 definitions of disinformation, 5 of 10 definitions of fake news, 6 of 24 definitions of infodemic, and 0 of 2 definitions of malinformation. Infodemic had 19 of 24 definitions addressing "information overload" and malinformation had 2 of 2 definitions with "accurate" and 1 definition "used in the wrong context." Out of all the definitions, 56 (75%) were referenced from other sources. CONCLUSIONS: While the definitions of misinformation and related terms in the health field had inconstancies and variability, they were largely consistent. Inconstancies related to the intentionality in misinformation definitions (7 definitions mention "unintentional," while 5 definitions have "intentional"). They also related to the content of infodemic (9 definitions mention "valid and invalid info," while 6 definitions have "false/inaccurate/incorrect"). The inclusion of concepts such as "intentional" may be difficult to operationalize as it is difficult to ascertain one's intentions. This scoping review has the strength of using a systematic method for retrieving articles but does not cover all definitions in the extant literature outside the field of health. This scoping review of the health literature identified several definitions for misinformation and related terms, which showed variability and included concepts that are difficult to operationalize. Health practitioners need to exert caution before labeling a piece of information as misinformation or any other related term and only do so after ascertaining accurateness and sometimes intentionality. Additional efforts are needed to allow future consensus around clear and operational definitions.

Proposed triggers for retiring a living systematic review.

Living systematic reviews (LSRs) are systematic reviews that are continually updated, incorporating relevant new evidence as it becomes available. LSRs are critical for decision-making in topics where the evidence continues to evolve. It is not feasible to continue to update LSRs indefinitely; however, guidance on when to retire LSRs from the living mode is not clear. We propose triggers for making such a decision. The first trigger is to retire LSRs when the evidence becomes conclusive for the outcomes that are required for decision-making. Conclusiveness of evidence is best determined based on the GRADE certainty of evidence construct, which is more comprehensive than solely relying on statistical considerations. The second trigger to retire LSRs is when the question becomes less pertinent for decision-making as determined by relevant stakeholders, including people affected by the problem, healthcare professionals, policymakers and researchers. LSRs can also be retired from a living mode when new studies are not anticipated to be published on the topic and when resources become unavailable to continue updating. We describe examples of retired LSRs and apply the proposed approach using one LSR about adjuvant tyrosine kinase inhibitors in high-risk renal cell carcinoma that we retired from a living mode and published its last update.

Testing guidelines during times of crisis: challenges and limitations of developing rapid and living guidelines.

BACKGROUND: The start of the COVID-19 pandemic presented a situation in which there was an urgent need for decision-making that relates to diagnosis, but the evidence was lacking, of low certainty or constantly changing. Rapid and living guideline development methods were needed and had to be applied to rigorous guideline approaches, such as the Grading of Recommendations Assessment, Development, and Evaluation approach. OBJECTIVES: To describe the process of developing rapid diagnosis guidelines when there is limited and imperfect available data at the time of crisis. SOURCES: Case example from four Infectious Disease Society of America COVID-19 diagnostic guidelines. CONTENT: As the world was experiencing panic with COVID-19, there were serious doubts about the feasibility of following a rigorous process for guideline development when timeliness was of extreme value. The Infectious Disease Society of America guideline panels supported by several methodologists strongly believed that at times of crisis, it is more important than ever to follow a rigorous process. The panel adopted a rapid and living systematic review methodology and applied the Grading of Recommendations Assessment, Development and Evaluation approach to four diagnosis guidelines despite the challenges of scarce and dynamic evidence. We describe the methodological details of the rapid and living approach (data extraction, meta-analysis, Evidence to Decision framework, and recommendation development), the challenge of resources, the challenge of scarce evidence, the challenge of rapidly changing evidence, as well as 'wins' from the Infectious Disease Society of America experience. IMPLICATIONS: Mitigation of pandemics relies on rapid and accurate diagnosis, which is challenged by many knowledge gaps. This necessitates emerging evidence is rapidly incorporated in a living fashion with several decisional and contextual factors to ensure the best public health strategies and care for patients. This process must be systematic and transparent for developing trustworthy guidelines and should be supported by all stakeholders, including researchers, editors, publishers, professional societies, and policymakers.

The life and death of living systematic reviews: a methodological survey.

OBJECTIVES: The objectives of this study are to describe the characteristics of living systematic reviews (LSRs) and to understand their life cycles. STUDY DESIGN AND SETTING: We conducted a comprehensive search up to April 2021 then selected articles and abstracted data in duplicate and independently. We undertook descriptive analyses and calculated delay in version update and delay since the last published version. RESULTS: We included 76 eligible LSRs with a total of 279 eligible versions. The majority of LSRs was from the clinical field (70%), was COVID-19 related (63%), and had a funding source specified (62%). The median number of versions per LSR was 2 (interquartile range (IQR) 1-4; range 1-19). The median and IQR for the ratio of the actual period of update to the planned period of update was 1.12 (0.81; 1.71). Out of all reviews with a 'planned period of update' and at least one update (N = 19), eight LSRs (42%) had a period since last published version greater than 3 times the planned period of update. No LSR included a 'retirement notice' in their latest published version. CONCLUSION: While most LSR complied with the planned period of producing updates, a substantive proportion lagged since their last update.

The Infectious Diseases Society of America Guidelines on the Diagnosis of COVID-19: Antigen Testing.

BACKGROUND: Immunoassays designed to detect SARS-CoV-2 protein antigens (Ag) are commonly used to diagnose COVID-19. The most widely used tests are lateral flow assays that generate results in approximately 15 minutes for diagnosis at the point-of-care. Higher throughput, laboratory-based SARS-CoV-2 Ag assays have also been developed. The number of commercially available SARS-CoV-2 Ag detection tests has increased rapidly, as has the COVID-19 diagnostic literature. The Infectious Diseases Society of America (IDSA) convened an expert panel to perform a systematic review of the literature and develop best practice guidance related to SARS-CoV-2 Ag testing. This guideline is an update to the third in a series of frequently updated COVID-19 diagnostic guidelines developed by the IDSA. OBJECTIVE: The IDSA's goal was to develop evidence-based recommendations or suggestions that assist clinicians, clinical laboratories, patients, public health authorities, administrators and policymakers in decisions related to the optimal use of SARS-CoV-2 Ag tests in both medical and non-medical settings. METHODS: A multidisciplinary panel of infectious diseases clinicians, clinical microbiologists and experts in systematic literature review identified and prioritized clinical questions related to the use of SARS-CoV-2 Ag tests. A review of relevant, peer-reviewed published literature was conducted through April 1, 2022. Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology was used to assess the certainty of evidence and make testing recommendations. RESULTS: The panel made ten diagnostic recommendations. These recommendations address Ag testing in symptomatic and asymptomatic individuals and assess single versus repeat testing strategies. CONCLUSIONS: U.S. Food and Drug Administration (FDA) SARS-CoV-2 Ag tests with Emergency Use Authorization (EUA) have high specificity and low to moderate sensitivity compared to nucleic acid amplification testing (NAAT). Ag test sensitivity is dependent on the presence or absence of symptoms, and in symptomatic patients, on timing of testing after symptom onset. In contrast, Ag tests have high specificity, and, in most cases, positive Ag results can be acted upon without confirmation. Results of point-of-care testing are comparable to those of laboratory-based testing, and observed or unobserved self-collection of specimens for testing yields similar results. Modeling suggests that repeat Ag testing increases sensitivity compared to testing once, but no empirical data were available to inform this question. Based on these observations, rapid RT-PCR or laboratory-based NAAT remains the testing method of choice for diagnosing SARS-CoV-2 infection. However, when timely molecular testing is not readily available or is logistically infeasible, Ag testing helps identify individuals with SARS-CoV-2 infection. Data were insufficient to make a recommendation about the utility of Ag testing to guide release of patients with COVID-19 from isolation. The overall quality of available evidence supporting use of Ag testing was graded as very low to moderate.

A Framework for the Development of Living Practice Guidelines in Health Care.

BACKGROUND: Living practice guidelines are increasingly being used to ensure that recommendations are responsive to rapidly emerging evidence. OBJECTIVE: To develop a framework that characterizes the processes of development of living practice guidelines in health care. DESIGN: First, 3 background reviews were conducted: a scoping review of methods papers, a review of handbooks of guideline-producing organizations, and an analytic review of selected living practice guidelines. Second, the core team drafted the first version of the framework. Finally, the core team refined the framework through an online survey and online discussions with a multidisciplinary international group of stakeholders. SETTING: International. PARTICIPANTS: Multidisciplinary group of 51 persons who have experience with guidelines. MEASUREMENTS: Not applicable. RESULTS: A major principle of the framework is that the unit of update in a living guideline is the individual recommendation. In addition to providing definitions, the framework addresses several processes. The planning process should address the organization's adoption of the living methodology as well as each specific guideline project. The production process consists of initiation, maintenance, and retirement phases. The reporting should cover the evidence surveillance time stamp, the outcome of reassessment of the body of evidence (when applicable), and the outcome of revisiting a recommendation (when applicable). The dissemination process may necessitate the use of different venues, including one for formal publication. LIMITATION: This study does not provide detailed or practical guidance for how the described concepts would be best implemented. CONCLUSION: The framework will help guideline developers in planning, producing, reporting, and disseminating living guideline projects. It will also help research methodologists study the processes of living guidelines. PRIMARY FUNDING SOURCE: None.